This application claims benefit of Japanese Application No. Hei 11-218598 filed in Japan on Aug. 2, 1999, the contents of which are incorporated by this reference.
The present invention relates to image pickup apparatus, and more particularly relates to an image pickup apparatus having a function for correcting false signals and to method of correcting deteriorated pixel signal thereof.
There is a problem in conventional solid-state image pickup apparatus using CCD image pickup device that image quality is deteriorated due to smears which occur as a phenomenon peculiar to CCD image pickup devices when an intense light has been incident thereupon. In particular, such phenomenon occurs as charge not captured at its true charge accumulating region is caused to leak into the vertical transfer line across the potential barrier. If caused by an ordinary and steady light, such phenomenon occurs over a vertical transfer period. In the case of a spot light, for example, it results in a vertical stripe extended up and down therefrom. Also, if the image object includes a horizontally extended bright portion such as a scenic view containing the sky, a flare-like whitish image over the frame as a whole is the outcome.
A method such as one disclosed in Japanese patent application laid open Hei-7-67038 has been known as the method for correcting such phenomenon. Noticing the fact that a smear occurs substantially at the same level along the vertical direction due to the principle of its occurrence, the output level of vertical OB (optical black) pixels, i.e., an optically concealed accumulating pixel region outside the effective pixel region, is determined as a smear eliminating reference signal which is subtracted from the pixel output signals of an effective frame.
An occurrence of large smear component, however, results in a disadvantage of image quality deterioration due to the correction to the contrary in the case where smear correction is simply employed by using the above described prior-art technique, since there is a limit in the output range of the image pickup device to be actually used in a camera. This will be explained by way of a specific example as follows. Suppose a state where a vertical stripe-like smear due to an intense spot light occurs at a level reaching the saturation level (100%) of the image pickup apparatus. The output signals of the image pickup device of the portion without smear are outputted without change even after the smear correction. Since the device output at the portion of smear, however, stays at 100% due to the saturation of the image pickup device, the output thereof after the smear correction (device outputxe2x80x94smear component) becomes xe2x80x9c0xe2x80x9d. Accordingly a vertical black stripe occurs. At this time, if the spot light portion itself is regarded as an image object, xe2x80x9c0xe2x80x9d output is the outcome also in this portion.
Such vertically extended black stripe or the phenomenon of blacked state of a supposedly very bright object produces a strange image which is quite unnatural. This is an impermissible deterioration of image quality (failed image) for an image pickup apparatus for taking various object scenes, even though a normal image object with a small level of smear can be taken at a higher image quality as compared to the conventional example. Accordingly, there is a problem that the above described smear correction method is difficult to be employed.
In particular, if the smear reaches near the saturation level, it is hardly possible to obtain a luminance information of image object. Hence the prior-art correction is meaningless or produces new false signals to the contrary. This might be mentioned as the state of more harm than good. At least, it has been difficult to obtain an image quality which is not unnatural.
To solve the above described problems in the prior-art smear correction method, it is an object of the present invention to provide an image pickup apparatus and method of correcting deteriorated pixels thereof in which at least an unnatural deterioration of image quality that possibly results in a failure is prevented from occurring and, at the same time, an effective smear correction with improving practical image quality as compared to the prior art is possible also of those portions for which luminance information of image object cannot be obtained at all due to smears.
It is a further object of the invention to provide an image pickup apparatus and method of correcting deteriorated pixels thereof in which a determination can be made on the basis of the level of smear components in applying compensation of data to deteriorated pixels caused for example by saturation of signal which cannot be handled by the prior-art smear correction.
It is another object of the invention to provide an image pickup apparatus and method of correcting deteriorated pixels thereof in which a more complicated determination can be made by considering both the level of smear components and the output level of effective signal pixels in applying compensation of data to deteriorated pixels caused for example by saturation of signal which cannot be handled by the prior-art smear correction.
It is another object of the invention to provide an image pickup apparatus and method of correcting deteriorated pixels in which a possible generation of unnatural image as a result of unlimited performance of compensation is prevented.
It is another object of the invention to provide an image pickup apparatus and method of correcting deteriorated pixels thereof in which an image can be obtained with reducing or eliminating unnatural, conspicuous false signals typically represented by white stripes which are unavoidable according to thee principle thereof.
It is another object of the invention to provide an image pickup apparatus and method of correcting deteriorated pixels thereof in which an image can be obtained as generally of high quality where smears are mitigated and at the same time unnatural, conspicuous false signals are reduced or eliminated.
In accordance with a first aspect of the invention, there is provided an image pickup apparatus including: smear level detection means for detecting smear level occurring in effective pixels of an image pickup device on the basis of signals outputted from an optically shielded pixel array of the image pickup device in a vertical blanking period thereof; deteriorated pixel determination means for, in the case where the smear level detected at the smear level detection means exceeds a predetermined value, determining corresponding effective pixels as output deteriorated pixels; compensation condition determination means for, when signal from a pixel determined as output deteriorated pixel at the deteriorated pixel determination means is at a predetermined saturation level, determining whether or not the deteriorated pixel satisfies predetermined compensation conditions; and deteriorated pixel correction means for replacing signals from deteriorated pixels determined as satisfying the compensations conditions at the compensation condition determination means with signals of undeteriorated pixels neighboring the deteriorated pixels and for correcting output level of signals from deteriorated pixels not satisfying the compensation conditions in accordance with the smear level detected at the smear level detection means and the saturation level.
The smear level detection means and the deteriorated pixel determination means are thus provided so that, when the detected smear level exceeds a predetermined value, corresponding effective pixels are determined as output deteriorated pixels. Hence it becomes possible to apply compensation of data to pixels which cannot be handled by the prior-art correction due for example to saturation of signal. Further the determination at that time can be made by the smear level alone.
Further, since the compensation condition determination means is provided as for determining whether or not an output deteriorated pixel satisfies predetermined compensation conditions, it is possible to prevent a possible generation of an unnatural image as a result of unlimited compensation.
Furthermore, since the deteriorated pixel correction means is provided as having the above described construction, data compensation based on neighboring undeteriorated pixel signals is performed for those deteriorated pixels determined as satisfying the predetermined conditions. An image is thus obtained with reducing or eliminating unnatural, conspicuous false signals typically represented by white stripes which are unavoidable according to the principle. Moreover, since output level of those deteriorated pixels not satisfying the predetermined conditions is corrected in accordance with the smear level and saturation level, an image is obtained as of a higher quality considering both the smear level and saturation level where smears are mitigated and at the same time unnatural, conspicuous false signals are reduced or eliminated.
The smear level of the image pickup device detected by the smear level detection means in the first aspect of the invention is preferably formed by what is obtained by dividing the sum of signal levels from an optically shielded pixel column having the same horizontal address by the total number of pixels in the same column.
Further, the predetermined compensation conditions in the first aspect of the invention are preferably that at least one undeteriorated pixel column is included within the neighboring four pixel columns at the center of which the pertinent deteriorated pixel column is located.
Furthermore, correction of output level corresponding to the smear level by the deteriorated pixel correction means in the first aspect of the invention is preferably rendered such that the detected smear level is subtracted from a deteriorated pixel level and the result is multiplied by a coefficient based on the saturation level and detected smear level.
In accordance with a second aspect of the invention, there is provided a method of correcting deteriorated pixel signals of image pickup apparatus, including the steps of: detecting smear level occurring in effective pixels of an image pickup device on the basis of signals outputted from an optically shielded pixel array of the image pickup device in a vertical blanking period thereof; in the case where the smear level detected at the smear level detecting step exceeds a predetermined value, determining corresponding effective pixels as output deteriorated pixels; when signal from a pixel determined as output deteriorated pixel at the deteriorated pixel determining step is at a predetermined saturation level, determining whether or not the deteriorated pixel satisfies predetermined compensation conditions; and replacing signals from deteriorated pixels determined as satisfying the compensation conditions at the compensation condition determining step with signals of undeteriorated pixels neighboring the deteriorated pixels and correcting output level of signals from deteriorated pixels not satisfying the compensation conditions in accordance with the smear level detected at the smear level detecting step and the saturation level.
The smear level detecting step and the deteriorated pixel determining step are thus provided so that, when the detected smear level exceeds a predetermined value, corresponding effective pixels are determined as output deteriorated pixels. Hence it becomes possible to apply compensation of data to pixels which cannot be handled by the prior-art correction due for example to saturation of signal. Further the determination at that time can be made by the smear level alone.
Further, since the compensation condition determining step is provided as for determining whether or not an output deteriorated pixel satisfies predetermined compensation conditions, it is possible to prevent a possible generation of an unnatural image as a result of unlimited compensation.
Furthermore, since the deteriorated pixel correcting step is provided in the above described manner, data compensation based on neighboring undeteriorated pixel signals is performed for those deteriorated pixels determined as satisfying the predetermined conditions. An image is thus obtained with reducing or eliminating unnatural, conspicuous false signals typically represented by white stripes which are unavoidable according to the principle. Moreover, since output level of those deteriorated pixels not satisfying the predetermined conditions is corrected in accordance with the smear level and saturation level, an image is obtained as of a higher quality considering both the smear level and saturation level where smears are mitigated and at the same time unnatural, conspicuous false signals are reduced or eliminated.
Further, the smear level of the image pickup device detected at the smear level detecting step in the second aspect of the invention is preferably what is obtained by dividing the sum of signal levels from an optically shielded pixel column having the same horizontal address by the total number of pixels in the same column.
Furthermore, the predetermined compensation conditions in the second aspect of the invention are preferably that at least one undeteriorated pixel column is included within the neighboring four pixel columns at the center of which the pertinent deteriorated pixel column is located.
Moreover, correction of output level corresponding to the smear level at the deteriorated pixel correcting step in the second aspect of the invention is preferably such that the detected smear level is subtracted from a deteriorated pixel level and the result is multiplied by a coefficient based on the saturation level and detected smear level.